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ABSTRACT 


A combined  source-channel  coding  approach  is  described  for  the  encoding, 
transmission  and  remote  reconstruction  of  image  data.  The  source  encoder 
employs  two-dimensional  (2-D)  differential  pulse  code  modulation  (DPCM). 

This  is  a relatively  efficient  encoding  scheme  in  the  absence  of  channel 
errors.  In  the  presence  of  channel  errors,  however,  the  performance  degrades 
rapidly.  By  providing  error  control  protection  to  those  encoded  bits  which 
contribute  most  significantly  to  image  reconstruction,  it  is  possible  to 
minimize  this  degradation  without  sacrificing  transmission  bandwidth.  The 
result  is  a relatively  robust  design  which  is  reasonably  insensitive  to 
channel  errors  and  yet  provides  performance  approaching  the  rate-distortion 
bound.  Analytical  results  are  provided  for  assumed  2-D  autoregressive  image 
models  while  simulation  results  are  described  for  real-world  images. 
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I . Introduction: 

There  has  been  widespread  interest  recently  in  the  efficient  encoding 
of  image  data.  Two-dimensional  (2-D)  differential  pulse  code  modulation 
(DPCM)  is  a technique  which  has  been  widely  employed  for  this  purpose.  Its 
behavior  in  the  absence  of  channel  errors  has  been  studied  extensively  and 
is  well-documented  [ 1 3— [ 5 ] - In  the  presence  of  channel  errors,  however, 
the  behavior  of  2-D  DPCM  degrades  rapidly.  Indeed,  on  some  fairly  represent- 
ative channels,  increasing  the  quantization  ac curacy  degrades  rather  than 
improves  the  subjective  reconstructed  image  quality.  As  a result,  some  form 
of  error  control  protection  must  be  provided  if  high-quality  image  reconstruct- 
ion is  to  be  achieved.  Since  channel  coding  generally  entails  a bandwidth 
expansion,  this  operation  can  be  extremely  wasteful  of  channel  bandwidth 
unless  applied  judiciously.  In  particular,  tradeoffs  must  be  made  between 
the  reconstruction  accuracy  associated  with  the  source  coder  and  the  degree 
of  error  control  protection  provided  by  the  channel  coding. 

In  this  paper  we  describe  a combined  source- channel  coding  approach  for 
the  encoding,  transmission  and  remote  reconstruction  of  image  data  which  ex- 
ploits these  tradeoffs.  The  source  encoder  employs  2-D  DPCM  which  has  been 
appropriately  matched  to  the  image  source.  By  providing  selective  error 
control  protection  to  those  bits  which  contribute  most  significantly  to  image 
reconstruction,  it  is  possible  to  significantly  improve  the  reconstructed 
image  quality  without  sacrificing  transmission  bandwidth.  The  result  is  a 
relatively  robust  design  which  is  reasonably  insensitive  to  channel  errors 
and  yet  provides  performance  approaching  an  achievable  rate-distortion  bound. 
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The  overall  2-D  DPCM  system  is  described  in  Section  II  and  includes 
typical  performance  results  without  channel  coding.  An  analysis  of  the 
reconstructed  output  signal-to-noise  ratio  performance  is  provided  in 
Section  III.  Theoretical  performance  bounds  employing  channel  coding 
are  described  in  Section  IV.  In  particular,  it  is  shown  that  these  bounds 
indicate  performance  approaching  the  rate-distortion  bound  can  be  achieved 
with  properly  chosen  error  control  protection.  In  Section  V we  demonstrate 
that  these  performance  bounds  are  realistic  in  the  sense  that  they  can  be 
approximated  with  a readily  implemented  class  of  convolutional  codes. 
Simulation  results  are  provided  for  typical  real-world  images  in  Section  VI. 
Finally,  in  Section  VII  a summary  and  suggestions  for  future  research  are 
provided. 

II.  Preliminaries : 

While  there  are  a wide  range  of  possible  stochastic  models  for  images , 
we  will  restrict  attention  to  the  class  of  2-D  autoregressive  random  fields 
described  according  to 


K L, 

Si>j  ” jJ0  jJq  Si-k,j-Jl  + Wi,j 


(1) 


where  the  prime  is  intended  to  indicate  that  the  point  k=i.=0  is  excluded  from 
the  double  summation  and  {W.  } is  a 2-D  zero-mean  sequence  of  independent 
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and  identically  distributed  (i.i.d.)  random  variables  possessing  common 


variance  cr 


We  assume  that  the  initial  values  S for  k=0,l,  ...,  K-l 

“K  j“Xi 


and  i = 0,1,...,  L-l  have  been  specified.  These  initial  values  represent  the 
boundary  terms  as  illustrated  in  Fig.  1. 

As  a concrete  example  of  a 2-D  autoregressive  process  we  note  the  2-D 


Gauss-Markov  random  field.  Here 


with  Oi|p.|£l,  i=l,2  and^W  ) a 2-D  i.i.d.  zero-mean  Gaussian  sequence  possessing 
1 i * j 

common  variance  o2  = o2  (l-p?)(l-p§).  The  quantity  o2  represents  the  common  vari- 
w s * ^ s 

ance  of  the  resulting  sequence  {S.  } , i.e. , o2=var{S,  } , i,J>l.  It  is 

i.J  s i.J 

implicitly  assumed,  of  course,  that  the  initial  values  SQ  Q,  Sk  ^ and  SQ  ^ 
for  l£k^K  and  l<fc<L  have  been  appropriately  chosen  to  result  in  stationary 
conditions.  We  will  use  this  2-D  Gauss-Markov  random  field  as  an  illustrative 
example  in  what  follows. 

An  overall  block  diagram  of  the  DPCM  system  under  consideration  is  pro- 
vided in  Fig.  2.  Following  established  nomenclature,  the  DPCM  coding/decoding 

scheme  is  said  to  be  matched  to  the  message  model  if  a.  =a.  ..  That  is  the 

i.j  i.j  ’ 

coefficients  in  the  feedback  predictor  are  chosen  equal  to  the  2-D  auto- 
regressive coefficients  in  (l).  For  analysis  purposes  we  assume  this  to  be 
the  case  although  there  may  be  good  reason  for  other  choices. 

The  notation  of  Fig.  2 deserves  comment  at  this  time.  First  the  quantity 


Y.  , = S.  , + Q.  , 
i.j  i.J 


(3) 


represents  the  local  estimate  of  S.  , which  is  applied  as  input  to  the  feed- 

i . J 

back  predictor.  In  the  absence  of  channel  errors  this  is  identical  to  S^. 
the  reproduced  estimate  of  ^ released  to  the  destination.  The  error  in- 
curred in  this  case  is  identical  \,o  the  instantaneous  quantization  error  Q.  . 

1 » J 

In  the  absence  of  quantization  error  and  assuming  a.  =a.  , the  quantity 

1 . J 1.0 


.+  K L, 

3i">  ' Jo  Jo 


(U) 


represent  the  causal  least  mean-square  predicted  estimate  of  S.  making  use 

» J 

of  the  data  set  as  illustrated  in  Fig.  3.  Note  that  in  general  due  to  the 

quantization  errors  inherent  in  Y.  ,,  the  quantity  S,  , will  differ  from  the 

i > J 1 *d 
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causal  least  mean-square  predicted  estimate  of  S.  . While  it  would  be 

1 » J 

possible  to  use  past  values  of  the  sequence  {S.  } in  making  this  predict- 

1 » J 

ion  at  the  encoder,  this  information  is  unavailable  at  the  decoder.  In  order 

to  insure  that  both  the  encoder  and  decoder  utilize  the  same  information  and 

thus  avoid  an  otherwise  potentially  unstable  situation,  the  encoder  will  make 

~ + 

use  of  {Y  }in  forming  the  prediction  S.  . 

i > J ^ » J 

In  Fig.  2,  the  sequence  {X.  ,}  represents  the  transmitted  2-D  sequence 

i » J 

whose  values  assume  one  of  the  Q=2n  possible  output  levels  of  the  n-bit 

quantizer.  Exclusive  use  will  be  made  of  a symmetric  uniform  quantizer 

possessing  the  characteristic  illustrated  in  Fig.  1+.  Note  that  Eg=-“>  while 

E =<=°.  If  A represents  the  uniform  step  size  normalized  to  the  standard  devi- 

ation  a of  the  input  error  sequence  {E.  .}  then 

s i,J 

E - E = X.-X.  =Ao  (5) 

x.  Jo-1  x.  JO-1  e 

The  quantizer  output  level  X.  = [)0-(Q-l)/2]Aa  , JO=0,1, . . . ,Q-1  is  then  ex- 
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pressed  as  an  n-bit  binary  word  for  a subsequent  transmission  over  the  digital 
data  link.  In  what  follows  we  assume  that  a natural  binary  representation  is 
employed.  That  is,  the  output  level  X^  is  coded  into  the  natural  binary 
representation  of  the  integer  JO.  It  will  be  assumed  that  individual  bit 
streams  are  available  for  subsequent  modulation/coding  and  transmission.  In 
particular,  the  modulation/coding  is  allowed  to  be  different  for  each  of  the 


n serial  bit  streams  associated  with  the  output  of  the  n-bit  quantizer. 

In  all  cases,  the  quantizer  characteristics  are  chosen  to  minimize  the 
mean-square  quantization  error.  That  is,  the  single  parameter  A is  chosen  to 
minimize  the  quantity 

+ In  this  context  "past"  values  represent  elements  in  the  data  set  used  for 
causal  prediction  as  illustrated  in  Fig.  3. 


where  p„( • ) is  the  probability  density  function  (p.d.f.)  associated  with  the 

error  sequence  {E.  }.  It  is  extremely  difficult  to  provide  explicit  evalu- 

1 > J 

ation  of  p ( • ) in  any  realistic  situation.  As  a result,  we  restrict  attention 

Jjj 

to  the  two  cases  where  the  sequence  {E.  } is  assumed  either  Gaussian  or 

i > J 

Laplacian  distributed.  From  the  experimental  results  provided  in  [6]  there 

is  substantial  evidence  that  the  error  sequence  {E.  } is  approximately 

1 » J 

Laplacian  distributed  for  a wide-range  of  real-world  images.  For  the  2-D 
Gauss-Markov  random  field  described  by  (2),  on  the  other  hand,  we  would  expect 
the  error  sequence  to  be  approximately  Gaussian  distributed.  The  two  cases 
considered  then  span  a range  of  both  practical  and  theoretical  interest. 

Tabulations  of  the  optimum  A for  various  number  of  quantization  levels  Q have 
been  provided  for  the  Gaussian  distribution  in  Max  [7],  and  likewise,  for  the 
Laplacian  distribution  in  Paez  and  Glisson  [8].  For  convenience  these  results 
are  summarized  in  Table  1.  It  should  be  noted,  however,  that  the  design 
approach  employed  here  of  optimizing  the  quantizer  characteristics  to  mini- 
mize is  a decidedly  suboptimum  approach.  That  is,  the  quantizer  charact- 
eristics should  be  chosen  to  minimize  the  overall  reconstruction  error.  The 
mathematical  intractability  of  the  latter  approach  has  influenced  the  approach 
adopted  here. 

Finally  the  sequence  {R.  }appearing  at  the  channel  output  is  described 

i » J 

according  to 


Ri,J  = Xi,J+Ni,j 


(7) 


where  the  sequence  {N.  } represents  additive  digital  channel  noise  assumed 

1 >0 

independent  from  pixel-to-pixel.  The  reproducing  estimate  {S.  } delivered 

1 > J 

to  the  destination  is  then  determined  recursively  according  to 


(8) 
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®i.J  ~ J0  J0  ak,£Si-k,j-£+Ri,j  ; 1>J-° 

again  assuming  that  appropriate  initial  conditions  have  been  specified  for 
S , for  k=0,l, . . . ,K-1  and  £=0,1, . . . ,L-1.  As  noted  previously,  in  the 

“K  )"X> 

absence  of  channel  errors  (i.e.,  R.  =X.  .)  we  have 

' 5i,j  * qi,j  (9) 

so  that  the  only  source  of  error  is  the  quantization  noise.  The  indexing  of 

the  channel  output  sequence  {R.  } must,  of  course,  correspond  to  some  estab- 

i,J 

lished  scanning  pattern.  In  what  follows  we  assume  that  the  image  is  scanned 

in  a fixed  row-by-row  raster  scanning  pattern  and  the  sequence  {R.  } and 

i,3 

hence  {S.  } is  ordered  accordingly. 

i,3 

In  Fig.'s  5 and  6 we  illustrate  typical  behavior  of  this  2-D  DPCM  encod- 
ing scheme  on  a head-and-shoulders  image  for  an  optimum  uniform  Gaussian  and 
Laplacian  quantizer  respectively  both  in  the  absence  of  channel  errors.  Corres- 
ponding results  for  an  outdoor  scene  are  illustrated  in  Fig.'s  7 and  8.  The 
images  here,  as  in  the  remainder  of  the  paper, are  all  256  x 256  arrays. 

These  results  assume  a first-order  2-D  autoregressive  image  model  similar 

A + 

to  that  described  by  (2).  In  particular,  the  order  of  the  predictor  S. 

i » o 

in  (1*)  is  such  that  K=L=1.  Least-squares  estimates  of  the  vertical  and  hori- 
zontal correlation  coefficients,  and  respectively,  were  obtained  and 
the  predictor  feedback  coefficients  set  equal  to  the  corresponding  estimated 
quantities.  The  resulting  estimates  are  provided  in  Table  2 and  are  typical 
of  the  values  associated  with  a wide  range  of  real-world  images.  These  two 
test  images  will  be  used  for  purposes  of  illustration  in  the  remainder  of 
this  paper. 


In  the  absence  of  channel  errors  the  2-D  DPCM  encoding  scheme  provides 
reasonably  good  fidelity  even  for  1 bit/pixel  although  there  are  some  distor- 
tions particularly  in  the  vicinity  of  sharp  edges.  There  is  marginal  sub- 
jective improvement  beyond  2-3  bits/pixel.  Results  using  the  Laplacian 
quantizer  provide  barely  perceptible  subjective  improvement  indicating 
some  insensitivity  to  quantizer  choice.  In  what  follows,  all  displayed 
images  will  be  for  the  case  of  an  optimum  uniform  Laplacian  quantizer  unless 
explicitly  noted  to  the  contrary. 

The  effects  of  channel  errors  on  both  the  head-and-shoulders  image 

and  outdoor  scene  can  be  observed  in  Fig.'s  9 and  10 respectively . Here 

the  channel  is  modeled  as  a binary  symmetric  channel  (BSC)  with  bit  error 
_3 

probability  P^=10  . Subjective  tests  have  indicated  a sharp  threshold  at 

this  point  with  reconstructed  image  quality  degrading  rapidly  beyond  this 

point.  This  is  indicated  most  clearly  in  Fig.'s  11  and  12  indicating  results 

-U  -3 

for  the  outdoor  scene  at  P.  =10  and  P,  =5x10  respectively.  The  effects  of 

D D 

channel  errors  are  much  more  pronounced  as  the  number  of  quantization  levels 
Q increases.  Indeed,  for  these  channel  error  rates  it  would  appear  wasteful 
of  channel  bandwidth  to  employ  more  than  1-2  bits/pixel.  Increasing  the 
quantizer  accuracy  degrades  rather  than  improves  image  quality.  Similar  re- 
sults apply  in  the  case  of  a Gaussian  quantizer.  The  explanation  for  this 
behavior  can  be  seen  most  clearly  from  the  explicit  evaluation  of  output 
eignal-to-noise  ratio  provided  in  the  following  section.  We  demonstrate 
how,  by  a judicious  use  of  channel  coding,  the  subjective  image  quality  can 
be  improved  substantially  in  the  presence  of  channel  noise  without  sacrificing 
channel  bandwidth. 


III.  Analysis  of  Output  Signal-to-Noise  Ratio: 


The  output  signal-to-noise  ratio  (SNRQ)  of  the  2-D  DPCM  system  is  given 


by 


(°*/°*) 
SNR  = — - — — 
0 „2 


(10) 


where  a2  is  the  variance  of  the  assumed  zero-mean  stationary  2-D  sequence 
s 


{ S . } and 

i 9 J 


4 ■ Et'si,j-si,jj'v<’e 


(11) 


is  the  resulting  mean-square  reconstruction  error  normalized  to  the  variance  o* 

of  the  stationary  sequence  {E.  } appearing  as  input  to  the  quantizer.  The 

i > J 

quantity  (°2/o2)  has  an  interpretation  as  the  mean-square  signal  level  normal- 


ized to  the  mean-square  prediction  error  which,  if  we  neglect  the  effects  of 

(12) 


quantization  noise,  is  easily  shown  to  be  given  by 

rir  fir 

. _ ) = (i-)2 

s e 


k2TT'  J 

-IT  -IT 


Here  D(z^,z2)  is  defined  as  a function  of  the  complex  variables  Z-^jZ^  as 
D(z,  ,z0)  = 


1’2'  l-H(zlSz2) 

where  H(z^,z2)  is  the  transfer  function  of  the  feedback  predictor  in  the 

encoder.  From  (4)  this  quantity  is  given  by 

K L,  v , 

H(z  ,z  ) = l l ot  z z 
1 2 k=0  1=0  k**  1 2 

For  example,  in  the  case  of  the  2-D  Gauss-Markov  process  in  (2)  we  have 


a2/o2  = 
s e 


(13) 


(1*0 


(i-ff)d^ 


(15) 
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Following  a fairly  standard  development  [9  ]- [12] , the  quantity 
in  (10)  can  be  expressed  as  the  sum  of  three  separate  and  conveniently 
normalized  components  according  to 


Here , 


em  = z + 2e  + e 
T q me 


e = E{Q.  }/a2  = a2/a2 
q i,j  e q e 


(16) 


represents  the  mean-square  value  of  the  quantization  noise  normalized  to  a2. 
Similarly,  the  quantity  represents  the  normalized  mean-square  error 
contribution  due  solely  to  channel  errors  and  evaluated  according  to 


e = E{N!2  }/o2 
c i,j  e 


(18) 


where  the  stationary  sequence  {N!  ,}  represents  the  additive  noise  at  the 
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output  of  the  DPCM  decoder  in  response  to  the  channel  noise  sequence  {N.  } 

i 9 J 

at  its  input.  The  system  transfer  function  of  the  decoder  Dtz^z^)  has 

been  given  previously  by  (13).  Finally,  the  quantity  e represents  a mutual 

m 

error  term  given  by 


e = E{Q.  N!  }/a2  . 

m i,j  i,J  e 


(19) 


We  consider  evaluation  of  each  of  these  terms  separately. 

As  stated  previously,  the  quantizer  has  been  designed  to  minimize  the 
variance  a2  of  the  quantization  noise.  It  is  easily  shown  that 


Q/2-1  rl A 

= 2 l [y-U-; 


e = 
<1 


1/2 )A]  pE(y)dy+2 


(Jl-l)A 


[y-(Q-l)A/2]2pE(y)dy  (20) 


(Q/2-1) A 


where  pv(y)  = a p„(a  y)  is  a normalized  version  of  either  the  Gaussian  or 
£j  e JL  e 

Laplacian  p.d.f.  possessing  zero  mean  and  unit  variance.  This  expression  for 
is  easily  evaluated  numerically  for  various  values  of  Q with  A taken  from 
Table  1. 


’I 


Consider  now  the  error  term  e contributed  by  channel  error  effects  as 

c 

defined  by  (l8).  Assuming  again  that  the  channel  noise  sequence  {N.  } is 

1 » J 

2 

independent  from  pixel-to-pixel  with  mean- square  value  e^  we  have 

ec  = (en/0e)(fc)2  / / e^2)^  dAg  (21) 

-IT  -'IT 

where  D(z^,z  ) is  the  discrete  2-D  system  transfer  function  associated  with 
the  decoder  and 

< - -X  X (W2  pm*p»  • (22) 

This  is  merely  the  mean-square  error  incurred  if  was  transmitted  and  decoded 
as  the  level  x.  averaged  over  the  probability  of  all  such  error  events.  The 

tv 

quantity  is  the  conditional  probability  that  level  was  transmitted 

and  decoded  as  level  while  is  the  probability  of  transmitting  level  x • 

2 2 

It  is  more  illuminating  to  consider  the  normalized  quantity  en/°e  which  is 
easily  seen  to  reduce  to 

Sn/t7e  = A2  ^ ^ (k‘°2  W*  (23) 

n 6 k=0  £=0  Z 

In  Appendix  A it  is  shown  that  this  last  quantity  can  be  evaluated  according  to 

en/0e  = A2{  ^ Pb  (l_Pb  )p2l+[  1 Pb  (l_Pb  )2±]2}  (2l4) 

i=0  i i i=0  i i 

where  P^  , i=0,l, . . . ,n-l  is  the  bit  error  probability  associated  with  each  of 
i 

the  successive  bit  positions  in  the  n-bit  quantizer  output  word.  For  the  part- 
icular case  of  equal  bit  error  probability  P,  =P.  , i=0,l, . . . ,n-l  and  this  last 

b . b 
i 

expression  reduces  to 

e2/o2  = A2P  ( 1-P  ) { ( Q2_l ) / 3 + Pjl-Pj(Q-l)2}  (25) 


n e b'  b' 


b b' 


where  we  recall  Q=2  . In  either  case,  the  error  contribution  ec  is  readily 
evaluated  for  various  values  of  Q with  A chosen  from  Table  1 for  either  the 
Gaussian  or  Laplacian  quantizer  characteristic.  Again  in  the  particular  case 
of  the  2-D  Gauss-Markov  process  given  by  (2)  we  find 


10 


(26) 


L 


e2/o2 
n e 


e = 


c (l-P^Ci-Pg) 


p p 

with  e /o  given  by  either  (2U)  or  (25)  as  appropriate, 
n e 


Finally,  the  mutual  error  term  is  shown  in  Appendix  B to  be  express- 


ible in  the  form 


Q-l  n-1  . ..  5 /•*/■*  1A 

«'2“  jo  (1‘2Ii)P»>i2  1 I D<e  ,e  )a*i  "2  ’ <27> 


-TT-7T 


Here  » i=0,l, . . . ,n-l  represent  coefficients  of  the  binary  expansion  of  the 


integer  l according  to 


n-1 

l = l l.2X  ; l.  = 0,1 

i=0  1 1 


(28) 


while  P_  is  the  hit  error  probability  associated  with  the  transmission  of  the 
b . 

1 


i’th  bit  and  finally 


rEnJ., /a 


4 = 1+1  e{y-U-(Q-l)/2]A}p  E(y)dF  ; *=0,  l,...,Q-l 


/a. 


(29) 


is  the  average  normalized  quantization  error  given  that  the  t'th  level  was  transmitted. 
For  the  particular  case  where  the  bit  error  probability  is  constant  for  each 
bit  position  we  have 

Q-l 


m 


= -UAPb[  l U-(Q-l)/2}e^](|^-)2  [ \ Dfe^.e^^dXg  . (30 ) 


-7r— 7 r 


In  either  case,  the  quantity  is  readily  computed  as  a function  of  Q and 


the  channel  error  probability  for  various  image  models. 

For  the  particular  case  of  the  2-D  Gauss-Markov  sequence  described  by  (2) 
it  is  readily  seen  that 

Tf  D(ejXl,eJX2)  dXxdX2  = 1 (3l) 


(i-)2 

v2ir 


— IT— TT 
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which  simplifies  evaluation  of  e . This  component  has  been  found  to  be 

m 

negligible  compared  to  the  other  error  components  and  e^. 

In  Fig.'s  13  and  lU  we  illustrate  the  behavior  of  SNRQ  for  an  assumed 
2-D  Gauss-Markov  image  model  of  the  head-and-shoulders  image  and  outdoor 
scene  respectively.  Here  a Gaussian  uniform  quantizer  was  used  and  individual 
bits  were  transmitted  using  coherent  binary  phase-shift  keyed  (BPSK)  modula- 
tion over  an  additive  white  Gaussian  noise  (AWGN)  channel.  The  bit  error 
probability  in  this  case  is  given  by 


where 


i=0,l,. 


Q(x) 


,n-l 


(32) 

(33) 


x 

and  is  the  signal  energy  per  transmitted  binary  channel  symbol  while  N^/2 
is  the  double-sided  noise  spectral  density  in  watts/Hz. 

In  these  figures,  the  quantity  SNRQ  is  plotted  as  a function  of  the  channel 
signal-to-noise  ratio  on  a normalized  per  pixel  basis.  That  is,  if  N uses 
of  the  channel  are  employed  to  transmit  the  n-bit  quantizer  output  word  corres- 
ponding to  each  pixel  position  then  the  channel  signal-to-noise  ratio  is  de- 
fined according  to 


N-l 

SNR  = l (2E  /N  ) 
1 i=0  S1  U 


(310 


where  Esi,  i=0,l, . . . ,N-1  is  the  signal  energy  for  each  channel  use.  Actually, 
in  Fig.'s  13  and  llj  N=n  while  we  assume  constant  energy  so  that  Eg^=Es  , i=0,l, 
...,n-l.  It  will  prove  convenient  when  employing  channel  coding  to  allow  the 
generality  afforded  by  (3l*).  The  bandwidth  expansion  factor  is  then  N/n. 
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Also  included  in  Fig.'s  13  and  ll*  is  the  rate-distortion  bound  on  SNRQ 
for  operation  over  an  AWGN  channel.  This  is  obtained  by  equating  the  rate- 
distortion  function,  computed  under  a mean-square  fidelity  criterion,  to 
the  channel  capacity  and  solving  for  SNR^  as  a function  of  SNR^.  The  rate- 
distortion  function  for  a continuous-amplitude  2-D  Gaussian  sequence  has  been 
given  by  Stuller  and  Kurz  [13]  following  the  development  by  Berger  [14  ] in  the 
1-D  case.  In  particular,  the  rate-distortion  function  R(D)  has  the  para- 


metric form 


D = (ijj-)2  min{0vS(X1,X2)}  dX^dXg 


(35a) 


R(D)=(fc)2 


x S(X  X ) 

max{0,  — l°g2 g ) dX1dX2 


(35b) 


measured  in  units  of  bits/sample.  Here  S(X^,X2)  represents  the  2-D  discrete 
power  spectral  density  which  for  a 2-D  autoregressive  process  as  in  (l)  becomes 


s(x1,x2)= 


K L. 


I l \ i 

k=0  1=0  ’ 


e — jkX  ie — j *-X2 


In  the  particular  case  of  the  2-D  Gauss-Markov  process  this  reduces  to 


s(x1,x2)  = 


(1-pf )(l-p|)a2 

( l-2p ! cosX I +p  2 ) ( l_2p  2cos  X 2+p  2 ) 


A computer  program  for  the  numerical  evaluation  of  R(D)  has  been  constructed 
as  described  in  [15].  In  the  present  context,  we  have  SNRQ=a^/D  where  D is 
the  mean-square  distortion. 

The  capacity  of  the  AWGN  channel  with  signal  energy  Egi  is  given  by  the 
well-known  expression  [l6] 
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c-jlogjll* 


2E  . 

A]  (bits/channel  use) 


(38) 


so  that  the  maximum  output  SNR  for  N channel  uses  satisfies 

N 


„ , „ SNR . 

B(»s  sm?)=  2 1o«2  t1+  TT4 


(39) 


allowing  solution  for  SNR^  in  terms  of  SNR^ . 

Several  things  should  he  observed  from  Fig.'s  l3  and  1**.  In  the  first 
place,  note  the  sharp  threshold  effect  which  becomes  more  pronounced  as  the 
number  of  quantization  levels  Q increases.  For  SNR^  in  excess  of  approximate- 
ly 20dB  channel  errors  are  rare  and  the  performance  is  limited  solely  by 
quantization  noise.  An  increase  in  the  number  of  quantization  levels  results 
in  a commensurate  increase  in  SNRq.  For  smaller  SNR^  we  see  a complete 
reversal  of  this  behavior.  For  example,  at  SNR^=12dB  relative  performance, 
measured  in  terms  of  SNR^,  actually  improves  by  decreasing  the  number  of 
quantization  levels.  This  is  an  illustration  of  the  behavior  observed 
previously  in  Fig.'s  9 through  12.  It  should  be  noted  that  this  threshold 
behavior  occurs  at  error  probabilities  typical  of  that  provided  on  many 
commercial  and  military  data  links. 

The  second  observation  to  be  made  from  Fig.'s  13  and  l1*  is  that  this 
2-D  DPCM  scheme  is  relatively  inefficient  compared  to  the  rate-distortion 
bound.  In  the  next  section  we  develop  ultimate  performance  bounds  to  be 
achieved  through  channel  coding.  These  bounds  are  based  upon  information- 
theoretic  considerations  and  apply  to  all  classes  of  codes.  Results  indi- 
cate relative  performance  arbitrarily  close  to  the  rate-distortion  bound 
over  a useful  range  of  SNR..  Later  we  demonstrate  that  these  performance 
bounds  are  realistic  in  the  sense  that  they  can  be  closely  approximated 

with  an  easily  implemented  class  of  convolutional  codes, 
t We  assume  constant  signal  energy  so  that  Es^=Es,  i=0,l, . . . ,N-1. 
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IV.  Theoretical  Performance  Bounds: 


A useful  upper  bound  on  SNRQ  can  be  obtained  by  lover  bounding  the  bit 

error  probability  P , i=l,2, . . . ,n-l  associated  with  each  of  the  bits  in  the 

i 

n-bit  quantizer  output  word.  Consider  first  the  case  where  the  modulation/ 

coding  system  treats  each  bit  identically.  Recall  that  N uses  of  the  channel 

are  required  to  transmit  the  quantizer  output  word  so  that  the  normalized  code 

rate  in  bits  per  channel  use  is  R=n/N.  Under  the  AWGN  assumption,  the  channel 

capacity  is  given  by  (38)  with  E . = E , the  constant  energy  per  channel  use. 

si  s 

The  converse  to  the  coding  theorem  (cf.  [l6],  [IT])  then  provides  the  desired 
lower  bound  on  the  common  bit  error  probability  P . Following  the  particularly 
illuminating  development  in  McEliece  [l8],  at  rates  above  the  capacity  C of  the 
AWGN  channel,  the  bit  error  probability  achievable  by  any  rate  R code  is  bounded 
away  from  zero  by 

P^  > H_1(l-C/R)  (UO) 

D — 

where  H 1(»)  represents  the  inverse  of  the  binary  entropy  function 

H(x)  = - x log2x  - (l-x)log2(l-x)  ; 0<x<l  (1*1) 


At  rates  below  capacity  P^  can  be  made  negligibly  small.  For  purposes  of 

numerical  evaluation,  P^  will  be  computed  according  to  the  right-hand  side 

b 

of  (40)  for  R»C  and  we  set  P =0  for  R<C.  It  should  be  noted  that  this  approach  is 

b — 

similar  to  that  employed  by  Chase  et  al. ,[  19 ] who  observe  that  no  known  codes 

can  achieve  this  lower  bound  on  P.  and  even  for  R<C  infinitely  long  codes 

b 

are  required  to  achieve  arbitrarily  small  bit  error  probability.  Nevertheless, 


In  Fig.'s  15-18  we  illustrate  typical  performance  results  for  the  outdoor 
scene  employing  a fixed-rate  modulation/coding  approach.  Similar  results  hold  for 
the  head-and-shoulders  image.  In  each  of  these  figures,  the  total  number  of 
channel  uses  is  held  fixed  at  the  values  N=2,3A,5  respectively  while  the 
number  of  bits  used  in  the  quantizer  assume  the  values  n=l,2,...,N.  Fixed-rate 
R=n/N  codes  then  insure  common  transmission  bandwidth  requirements  associated 
with  all  curves  on  the  same  figure.  These  curves  indicate  clearly  the  nature 
of  the  tradeoffs  possible  between  quantization  accuracy  and  sensitivity  to 
channel  errors  at  a fixed  transmission  bandwidth.  The  coded  system  performance 
is  characterized  by  a reduction  in  saturation  level  SNRQ  but  with  a threshold 
occurring  at  significantly  lower  SNR^.  Indeed,  the  knee  of  this  threshold 
characteristic  occurs  in  all  cases  within  2-3dB  of  the  rate-distortion  bound. 

For  example,  from  Fig.  15  if  SNRq>_25cLB  is  required  at  SNR^  as  low  as  5dB  and 
allowing  only  two  channel  uses  to  transmit  each  pixel  level  a 1-bit  quantizer 
is  required  in  conjunction  with  a rate  R=l/2  code.  This  level  of  performance 
could  not  be  achieved  with  a 2-bit  quantizer  and  no  coding. 

While  the  fixed -rate  coding  approach  appears  to  offer  performance  approach- 
ing the  rate-distortion  bound,  it  should  be  noted  that  codes  of  fairly  unusual 
rates  like  R=2/3,  3A,  U/5,  etc.  are  required.  Relatively  little  is  known 
concerning  the  construction  and  properties  of  practical  codes  of  these  rates. 

In  Fig.'s  19-21  we  illustrate  performance  bounds  on  SNRQ,  again  for  the  out- 
door scene,  when  each  bit  in  the  quantizer  output  word  can  be  accessed  and 
separately  coded.  Attention  is  restricted  to  code  rates  of  R=l/2, 1/3, lA 
and  1/5  since  a variety  of  practical  codes  exist  at  these  rates.  Again  each 
of  these  figures  corresponds  to  a fixed  number  of  channel  uses  per  pixel. 
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Error  control  protection  is  applied  only  to  the  most  significant  bit(s)(MSB) 
since  errors  here  contribute  most  heavily  to  reconstructed  image  quality.  For 
example,  in  Fig.  1 9 the  number  of  channel  uses  per  pixel  is  fixed  at  N=3.  The 
several  tradeoffs  include  a 3-bit  uncoded  system,  a 2-bit  system  with  a R=l/2 
code  on  the  MSB  with  the  remaining  bit  unprotected,  and  finally  a 1-bit  system 
with  a R=l/3  code  employed  on  the  only  bit.  Examination  of  these  figures 
indicates  the  relative  weakness  of  this  approach  compared  to  the  fixed  rate 
systems.  In  particular,  unless  some  degree  of  error  protection  is  applied 
to  all  bits  of  the  quantizer  output  word,  the  threshold  continues  to  exist 
at  uncomfortably  large  values  of  SNR^  and  the  performance  is  relatively  poor 
compared  to  the  rate-distortion  bound.  This  behavior  is  due  to  the  residual 
mean-square  error  contributions  from  unprotected  bit  errors. 

V.  Practical  Channel  Code  Performance: 

The  advantages  of  fixed  rate  channel  coding  in  conjunction  with  2-D  DPCM 
source  coding  have  been  illustrated  in  the  preceding  section.  Restriction  to 
practical  code  rates,  however,  generally  resulted  in  unequal  error  protection 
to  each  of  the  bits  associated  with  a quantizer  output  word.  Nevertheless, 
this  latter  approach  can  be  quite  effective  in  recovering  most  of  the  perform- 
ance gains  associated  with  the  fixed  rate  scheme.  It  remains  to  demonstrate 
the  degree  to  which  these  ultimate  performance  bounds  can  be  approached  by 
practical  coding  schemes.  In  particular,  we  restrict  attention  to  short 
constraint  length  convolutional  codes  of  rates  R=l/2,  1/3  and  l/H  decoded 
using  the  Viterbi  algorithm »an  excellent  description  of  which  can  be  found 
in  [20].  The  optimum  binary  codes  tabulated  by  Odenwalder [ 21 ] and  Larsen 
[22]  are  used  exclusively  in  what  follows. 
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A useful  exponentially  tight  bound  on  bit  error  probability  obtained 
with  Viterbi  decoding  on  memorylese  channels  can  be  expressed  in  the  general 
form 


Pb± 


dT(N,P) 


cLN 


N=l,D=Dr 


(us) 


where  T(D,N)  is  the  code  generating  function  (cf.  [20]  for  details)  while  the 


constants  KQ  and  depend  upon  the  particular  code  employed,  the  modulation 
strategy  in  use  and  the  channel  parameters.  For  example,  for  BPSK  modulation 
on  the  AWGN  channel  we  have 

f2dJ^  \ I d Jffi.  I 

exp  l4-^\  (U3a) 


while 


N„ 


Dq  = exp 


(U3b) 


where  d^  is  the  free  distance  of  the  code,  R is  the  normalized  code  rate  in 
information  bits  transmitted  per  channel  use  and  finally  E^/Nq  is  the  energy 
per  information  bit  normalized  to  the  single-sided  noise  spectral  density  NQ 
watts/Hz.  The  quantity  E^  is  related  to  the  energy  per  channel  symbol  Eg 
according  to  E^RE^.  A computer  program  has  been  developed  [23]  allowing 
numerical  evaluation  of  this  bound  and  thus  explicit  evaluation  of  SNRQ. 

Typical  results  are  illustrated  in  Fig.'s  22  for  the  case  N=2  channel  uses 
per  pixel  where  optimum  codes  of  constraint  length  K=3,  6 and  9 have  been 
employed.  Similar  results  are  provided  in  Fig.  23  for  the  case  N=3.  These 
figures  correspond  to  the  optimum  code  results  in  Fig.'s  15  and  19  respectively. 
The  convergence  of  the  practical  code  performance  toward  the  optimum  perform- 
ance with  increasing  constraint  length  K is  illustrated  clearly  in  Fig.'s  22 
and  23.  Unfortunately,  the  performance  even  for  a K=9  code  falls  short  of 
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pr*FT 


the  optimum  code  performance  by  some  2-3dB  and  slightly  more  when  compared 
to  the  rate- distort ion  bound. 

Also  included  in  Fig.'s  22  and  23  is  a dotted  curve  labeled  R*  bound  which 
is  displaced  some  2-3dB  from  the  rate-distortion  bound.  We  feel  that  this 
quantity  provides  a more  realistic  bound  on  achievable  performance  than  the 
rate  distortion  bound.  In  particular,  it  is  well-known  (cf.  [ll],  [2^])  that 
there  exists  a block  code  of  rate  R and  block  length  N such  that  the  probability 
of  error  in  decoding  a source  word  of  length  K=NR  is  bounded  according  to: 


P <2-N^R0-R^  ; R<R 


(HU) 


where  R^  is  the  so-called  critical  rate  associated  with  the  modulator/channel/ 
demodulator  cascade.  Thus,  for  block  codes,  the  single  number  RQ  provides  a 
measure  of  both  a range  of  rates  R for  which  reliable  communication  is  possible 
as  well  as  the  coding  complexity,  as  reflected  by  N,  required  to  guarantee  a 
specified  block-error  probability.  More  recently,  Viterbi  [25]  has  shown  for 
convolutional  coding  and  maximum-likelihood  sequence  decoding  on  the  constant, 
discrete  memoryless  channel  that  the  error  probability  is  upper  bounded  accord- 
ing to 


V-R 

P <C„  L 20  if  R<R. , 
e—  R 0 


(U5) 


where  K is  the  constraint  length  of  the  convolutional  code,  R is  the  code  rate, 
L is  the  total  number  of  source  letters  encoded,  and  C is  a weakly  dependent 

I\ 

function  of  R and  not  a function  of  L and  K.  Thus,  as  with  block  codes,  the 
single  number  RQ  provides  a measure  of  both  reliable  rates  and  code  complexity. 
Massey  [26],  [27]  has  used  these  observations  to  make  an  eloquent  and  persuas- 
ive argument  for  adopting  R^  as  a modulator-demodulator  design  parameter  in 
place  of  the  more  commonly  used  error  probability. 
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As  Wozencraft  and  Jacobs  [2U]  demonstrate,  the  quantity 

log  e , , 

R*  = ^^-[1+Eg/N0  - A+(Es/Nq)2]+  | log2[  | (1+  /1+(Es/Nq)2)]  (1*6) 

provides  a useful  upper  bound  on  R^  on  the  AWGN  channel  for  any  choice  of 
modulator-demodulator  parameters.  The  parameter  R^C  then  provides  a more 
realistic  limit  than  the  channel  capacity  C on  the  range  of  rates  achievable 
on  the  AWGN  channel.  In  Fig.'s  22  and  23  the  curve  labeled  R*  bound  then  is  obtain- 
ed by  equating  R*  to  the  rate-distortion  function  R(D)  and  solving  for  SNR^  as  a 
function  of  SNR^.  We  see  then  that  the  use  of  practically  implemented  short 
constraint  length  convolutional  codes  results  in  performance  within  l-2dB  of 
the  "practically  achievable"  R*  bound.  Similar  conclusions  can  be  drawn  from 
Fig.'s  2h  and  25  corresponding  to  N=U  and  5 channel  uses  respectively.  Here 
results  are  shown  only  for  practical  K =6  codes. 

VI.  Simulation  Results: 

The  effectiveness  of  the  combined  source-channel  coding  scheme  employing 
selected  constraint  length  K=6  convolutional  codes  are  illustrated  in  the  case 
of  the  outdoor  scene  by  the  simulation  results  provided  in  Fig.'s  26-29  for 
N=2,3>1*  and  5 respectively.  Corresponding  analytical  performance  evaluation 
is  illustrated  by  the  respective  curves  in  Fig.'s  22-25.  Similar  results  for 
the  head-and-shoulders  image  in  the  case  N=3  channel  uses  per  pixel  are 
illustrated  in  Fig. 30  . In  all  these  figures,  SNR^  is  constant  across  a row 
while  the  number  of  quantization  bits  n is  held  constant  for  each  column. 

The  transmission  bandwidth  requirements,  as  indicated  by  the  number  N of 
channel  uses  per  pixel,  is  constant  for  all  reconstructed  images  on  a figure. 
Representative  choices  for  SNR^  in  each  case  were  made  just  below  and  above 
pronounced  thresholds  in  the  SNRQ  curves  described  in  the  preceding  section. 


I 
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These  figures  illustrate  the  dramatic  improvements  in  subjective  recon- 


structed image  quality  resulting  from  a judicious  tradeoff  between  quanti- 
zation accuracy  and  error  control  protection  in  a fixed  transmission  band- 
width. For  example,  in  Fig.  28  we  see  clearly  the  advantages  in  allocating 
excess  transmission  bandwidth  to  error  control  rather  than  in  attempting  to 
improve  the  quantization  accuracy  by  increasing  the  number  of  quantization 
levels.  In  this  example,  the  2-bit  quantizer  employing  a K=6,  R=l/2  code 
on  each  bit  provides  far  superior  reconstructed  image  quality  than  the  ^-bit 
uncoded  system  or  even  the  3-bit  system  with  error  control  protection  on 
only  the  MSB. 

VII.  Summary  and  Conclusions: 

We  have  described  an  approach  to  the  combined  source-channel  coding  of 
a particular  class  of  image  sources.  This  approach  offers  theoretical  per- 
formance approximating  the  rate  distortion  bound.  We  have  shown  how  this 
theoretical  performance  can  in  turn  by  approximated  by  a class  of  practical 
short  constraint  length  convolutional  codes  and  demonstrated  the  performance 
gains  through  selected  simulations  on  real-world  images.  The  major  deficiency 
lies  in  the  lack  of  good  high  rate  code  constructions  required  to  achieve  the 
near  optimum  behavior  indicated  by  the  fixed- rate  code  performance  results. 

More  specifically,  we  have  been  forced  to  utilize  previously  tabulated  optimum 
codes  for  rates  R=l/2,  1/3  and  1/U.  This  severely  compromises  the  flexibility 
of  the  approach  in  those  situations  where  extremely  high  quality  image  recon- 
struction is  required,  i.e.,  high  saturation  level  SNR^  at  low  SNR^.  For  more 
moderate  reconstructed  image  quality  requirements  this  restriction  to  available 
codes  does  not  present  a problem  as  demonstrated,  for  example,  by  Fig.  2U  and  the 
corresponding  simulation  results  in  Fig.  28.  The  result  then,  at  least  for 
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moderate  reconstructed  image  quality  requirements,  is  a relatively  robust 
design  which  offers  a near  optimum  compromise  between  quantization  accuracy 
and  sensitivity  to  channel  errors  while  preserving  a fixed  transmission  band- 
width requirement.  Work  is  in  progress  to  determine  and  tabulate  the  proper- 
ties of  good  short  constraint  length  convolutional  codes  operating  at  the  re- 
latively high  rates  of  R=3/5,  2/3,  3A,  etc.  This  should  extend  the  flexibil- 
ity of  the  approach  considerably. 

Another  issue  which  deserves  attention  is  the  question  of  image  modeling. 
We  have  assumed  that  the  images  of  interest  could  be  modeled  as  2-D  auto- 
regressive models.  Indeed,  numerical  results  were  provided  only  for  the 
first-order  2-D  Gauss-Markov  model.  The  assumption  of  autoregressive  behavior 
with  Gaussian  statistics  is  a gross  oversimplification.  Similar  considerations 
apply,  however,  to  more  general  stochastic  image  models  and  this  aspect  is 
presently  under  investigation. 

Finally,  it  should  be  noted  that  we  have  made  exclusive  use  of  a uniform 
quantizer.  This  has  certainly  simplified  the  analysis  but  does  raise  the 
question  of  whether  significant  improvement  can  be  realized  through  improved 
quantizer  design.  We  feel  that  only  marginal  improvements  can  be  expected  by 
redesign  of  the  quantizer  although  this  question  is  being  explored. 


Evaluation  of  Normalized  Mean-Square  Error  Due  to  Channel  Noise 


The  normalized  mean-square  error  due  solely  to  channel  noise  is  given 
by  Eqn.  (23)  of  the  text  which  is  repeated  below  for  convenience 

Q-l  Q-l 

ej°e  = * l l (k4)2  P | P (A-l) 

n e k=0  i=0 

where  each  of  the  terms  has  previously  been  defined.  By  interchanging  the 
order  of  summation, the  summation  on  k is  easily  evaluated.  In  particular, 

p 

this  summation  is  recognized  as  the  conditional  mean  of  quantity  (k-£.) 
given  that  the  level  Z was  transmitted.  That  is, 

E{(k-«.)2|«.}  = ^ (k-H)2P  | = k2  -2k£  + Z2  (A-2) 

k=0  * 

where,  for  convenience,  the  overbar  indicates  conditional  expectation  given 
the  level  Z was  transmitted.  We  evaluate  each  of  these  terms  separately. 
Consider  first  the  quantity  k.  The  integer  k can  be  expressed  as 


n-1 

k = I k . 2 ; k-0 ,1, . . . ,Q-1 

i=0 

(A-3) 

where  k.=0,l. 
i 

Observe  that 

n-1 

k = l k 21 
i=0 

(A-U) 

while 

k,  = (1-P.  )A.  + P.  (l-£.)  ; i=0,l, . . . ,N-1 

l b . l b . l 

(A-5) 

1 1 


where  £^=0,1  is  the  corresponding  element  in  the  binary  expansion  of  the 

integer  Z representing  the  transmitted  signal  level  and  P is  the  bit-error 

i 

probability  associated  with  the  i'th  bit  which  is  allowed  to  be  different  for 
each  bit  position.  It  follows  then  that 


23 


(A-6) 


k = l { (l-2P  )fci+Pb  J21 
i=0  Di  1 

n-1  . 

= l + l (l-2Ai)Pb  2 


i=0 


This  shows,  in  particular,  that  there  is  a bias  introduced  in  the  declaration 
of  channel  output  level  due  to  channel  errors.  In  the  particular  case  when 


the  bit  error  probabilities  are  all  equal  so  that  P =P^  , i=0,l, . . . ,N-1, 

i 

we  have 

k = (l-2PbH  +(Q-l)Pb 


(A-7 ) 


where  it  is  recalled  that  Q=2  . 


Now  consider  the  quantity 
— n-1  n-1 


k = I l k-k-  2 

L L l j 


i+j 


i=0  j=0 


; k=0,l, . . . ,Q-1 


(A-8) 


If  i#j  we  have  k.k.  = k.k.  since  successive  bit  transmissions  are  assumed 
i j i i 

independent . For  i= j , on  the  other  hand , we  have 

k?  = (1-PV  )£?  + P^  (1-Jt?)=(l-2F  U.+P,_ 

l b.  l b.  l b.ib. 

li  li 

2 

where  we  have  made  use  of  the  fact  £.=£..  It  follows  that 

l l 


~ n-1 


■ I U1-2P  >VV2 


2i 


i=0 
n-1  n-1 


+ l l U1-2P  H,+P.  H (1-2P  )i  +P  }2 
i=OJ=0  bi  1 bi  bJ  J bJ 

i/j 


i+j 


(A-9) 


(A-10) 


whereupon  after  some  algebra  this  reduces  to 

k2  = l P (1-P  ) 221  + k2 

. 0 D.  D. 

1=0  1 1 


(A-ll) 


with  k given  by  (A-6).  In  the  particular  case  P,  =P^ , i=0,l, . . . ,N-1,  this 

b . b 

o 1 

yields  — P (1-P ,)(Q-1)  „ 

k2  = — \ + [ (l-2P  )t+(Q-l)P  ]2  • (A-12) 

3 b b 
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Finally,  in  the  general  case  we  have 

9 n_1  Oi 


n-1 

E{(k-a)^|t}  = l P (l-P  )2^  + 


i=0  V bi 


"f  (!-!>„  121 

1=0  l l 


which  is  independent  of  the  transmitted  level  l.  It  follows  that 


n-1 


en/<7f  = *2\  l Pb  (1-Pb  )p2i  + 

n S ’ i=0  bi  bi 


n-1 

l Pb  (l"Pb  )2" 

i=0  i Di 


12 


In  the  particular  case  that  Pb  = P^,  i=0,l,. . . ,N-1  we  have 

r 


= A2P  (l-p  ){ 
n e d b 


+ pbd-pb)(Q-D2! 


(A-13) 


(A-lU) 


(A-15) 
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Appendix  B 


Evaluation  of  Mutual  Error  Term 


The  mutual  error  term  e can  be  expressed  as 

m 


E{Q.  ,N!  ,}  o ” 

£m=  a2  °e  E|kL  *L  VA.A-k.J-* 


(B-l) 


where  {N.  } is  the  channel  noise  appearing  at  the  input  to  the  decoder  and 

1 > J 

{d^  is  the  2-D  impulse  response  sequence  of  the  decoder  evaluated  according 

V*  = {A)2^D(zl’Z2)z^"lz2‘ldzldz2  (B-2) 


with  the  closed  contour  integral  taken  around  the  unit  polydisc  in  the  complex 
space  c2  possessing  coordinates  z , z^.  Assuming  the  channel  is  memoryless 


it  follows  that 

E{Qi,/k,lt}  = 6i-k,j-£ 

with  6.  the  2-D  impulse  sequence  defined  according  to 
i 9 J 

6 = ( 1 5 i=J=° 

I 0 ; elsewhere 

Evaluation  of  the  expectation  on  the  right-hand  side  of  (B-l)  yields 


E{Q.  N . .} 


E(Q,  ,N.  4} 


(B-3) 


(B-U) 


(|^)2  f D(eJAl,ejX2)dX1dX2  . (B-5) 

n J 


Observe  now  that 


E{Q.  , N.  } P Q-l  Q-l  [ rE  , 

'i  =a~  l I (C-X  )p  (Odcl  (X^- 

q k=0  1=0  | | J * 


XJl^PkU 


Q-l  Q-l 

- A , L A (k-*)e*pku 


k=0  2 = 0 


(B-6) 
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where 


•i* 


'Et-l/0e 


{y-[A-(Q-l)/2]A}pE(y)dy  ; 4=0,1, • • • ,0-1  • (B-T) 


V°e 


Furthermore,  since  p (•)  is  assumed  symmetric  it  is  easily  established  that 

£ 

Ei  = " %l-l  1 *=0,l,...,Q/2-l  . (B-8) 

The  sum  on  k appearing  on  the  right-hand  side  of  ( B— 6 ) is  easily  recog- 
nized as  the  conditional  expectation  of  the  difference  (k-4)  given  that  the 


4'th  level  was  transmitted.  That  is 

Q-l 


E{(k-4)|4}  = I (k-4)p  | = k - 4 

k=0  K|i6 


(B-9) 


where,  as  in  Appendix  A,  the  overbar  indicates  conditional  expectation  given 


the  4'th  level  was  transmitted.  From  previously  established  results 

n-1 

E{ (k-4)  1 4}  = l (1-24  )P  21 

i=0  l 


(B-10) 


where  again  J i=0,l, . . . ,n-l  represent  coefficients  in  the  binary  expansion 

of  the  integer  4 and  P is  the  associated  bit  error  probability.  It  follows 

i 

that 


E(«,  ,N  ,}  Q-l  , 
4 i \ 

a 4=0 


n-1  , \ 

l (1-21  )P  2 

i-o  1 bi 


(B-ll) 


Now  consider  performing  the  summation  on  4.  For  each  4=0,1, ... ,Q/2-lthe  term 
within  braces  will  be  the  negative  of  that  for  the  index  Q-l-4.  From  the 
similar  symmetry  condition  expressed  by  (B-8)  we  have 


E{Q.  ,N  .}  Q-l  , 
l?~  ■Jlj  = 2A  l ^ 

o 4= Q/2 


T ( -^24  )P  2*  Ve 


i=0  1 bi 


(B-12) 


so  that  finally 


e = 2A 
m 


Q-l  i n-1 

I < l (l-2«i)Pb2  K 

«-Q/Q  \i=0  1 *i  J 


1-1 

(^)2 

f’f 

1 * 

J 

-7T-TT 

r* 


D(eJXl,eJX2)dX1dX2  • (B-13) 
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Number  of  Output 


Normalized  Step 


Levels  Q 


2 

4 

8 

16 

32 


Gaussian 


1.596 

0.996 

0.586 

0.335 

0.1881 


Size  4 
Laplacian 


1.414 

1.087 

0.731 

0.456 

0.281 


Table  1 

Summary  of  Normalized  Step  Size  A 
for  Optimum  Uniform  Quantizers  for 
both  Gaussian  and  Laplacian  Error 
Sequences 
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Figure  1 

Illustration  of  Image  and  Boundary 
Points  in  2-D  Autoregressive  Process 
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figure  3 

Illustration  of  Data  Set  Used  in 
Casual  Prediction  of  . 


+-elements  used 
in  prediction 
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2-bit  DPCM 
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e)  Wbit  DPCM  f)  5-tit  DPCM 

Figure  5 

2-D  DPCM  Encoding  of  Typical  Head-and-Shoulders  Image. 
Quantization  Effects  on  Reconstructed  Image;  Gaussian  Quantizer. 
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1-bit  DPCM 


3-bit  DPCM 


2-bit  DPCM 


e)  4-bit  DPCM 


5-bit  DPCM 


Figure  6 

2-D  DPCM  Encoding  of  Typical  I!ead-and-Shoulders  Image. 
Quantization  Effects  on  Reconstructed  Image;  Laplacian  Quantizer 
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e)  U-bit  DPCM 


5-bit  DPCM 


2-D  DPCM  Encoding  of  Typical  Outdoor  Scene. 
Quantization  Effects  on  Reconstructed  Image;  Gaussian  Quantizer 
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Figure  8 

2-D  DPCM  Encoding  of  Typical  Outdoor  Scene. 
Quantization  Effects  on  Reconstructed  Image;  Laplacian  Quantizer 
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Figure  9 

Channel  Error  Effects  on  2-D  DPCM  Image  Decoding. 
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Typical  Head-and-Shoulders  Image ;P  =10 
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Figure  10 

Channel  Error  Effects  on  2-D  DPCM  Image  Decoding. 
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Typical  Outdoor  Scene* P =10 
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Figure  11 


Channel  Error  Effects  on  2-D  DPCM  Image  Decoding. 
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Typical  Outdoor  Scene ;P^=10 


a)  Original 


b)  1-bit  DPCM 
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d)  3-bit  DPCM 
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Figure  12 

Channel  Error  Effects  on  2-D  DPCM  Image  Decoding. 
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Typical  Outdoor  Scene! P =5x10 
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Uncoded  2-D  DPCM  System  Performance  for 
Head-and-Shoulders  Image;  Gaussian  Quantizer. 
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Figure  16 

Performance  Bounds  for  Optimum  Fixed-Rate  Codes; 
Outdoor  Scene;  Gaussian  Quantizer. 
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Figure  19 

Performance  Bounds  for  Optimum  Practical-Rate  Codes 
Combined  with  No  Coding;  Outdoor  Scene;  Gaussian  Quantizer. 
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Performance  Bounds  for  Optimum  Practical-Rate  Codes 
Combined  with  No  Coding;  Outdoor  Scene;  Gaussian  Quantizer. 
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Figure  21 

Performance  Bounds  for  Optimum  Practical-Rate  Codes 
Combined  with  No  Coding;  Outdoor  Scene;  Gaussian  Quantizer. 
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Practical  Convolutional  Code  Performance; 
Outdoor  Scene;  Gaussian  Quantizer. 
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Figure  23 


Practical  Convolutional  Code  Performance; 
Outdoor  Scene;  Gaussian  Quantizer. 


5^ 


Practical  Convolutional  Code  Performance; 
Outdoor  Scene;  Gaussian  Quantizer 
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Figure  27 

Simulation  Results  on  Outdoor  Scene;  N=3  Bits  Transmitted. 
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Figure  30 

Simulation  Results  on  Head-and-Shoulders  Image \ N=3  Bits  Transmitted 
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